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ABSTRACT 

The effectivess of an improved static retraining method was 
evaluated for a simulated space vehicle approach and landing under 
instrument and visual flight conditions. Experienced pilots were 
trained and then tested after 4 months without" flying to compare 
their performance using the improved method with three methods 
previously evaluated. Use of the improved static retraining 
imethod resulted in no practical or significant skill degradation 
and was found tr' he even more effective than methods using a 
dynamic presentation of visual cues. The results suggested that 
properly structured open-loop methods of flight control task 
retraining are feasible. - - 


D180- 17876-1 


FOREWORD 

Tills report summarizes an experimental study accomplished as part of 
a program designed to investigate the degradation of learned skills 
as applicable to spaceflight tasks. The research reported here was 
begun in July 1973 and was completed in March 1974 for the NASA 
Lyndon B. Johnson Space Center under Contract NAS9-13550. The study 
was initiated by Dr. William E. Fedderson, Performance Section Head, 
Biomedical Research Division, Life Sciences Directorate. Dr. Fedder- 
son was the NASA Project Monitor throughout the study. ' 

The Boeing Program Manager and Principal Investigator was Dr. Thomas 
E. Sitterley. The author gratefully acknowledges the extensive assis- 
tance of Hr. Verle E. Helsel who was technical leader for advanced 
training method development, to Hr. Stephen Gough for his contribution 
in flight simulator computer operations, and to Mr. Allen Fukusliima 
for his engineering assistance in simulator, terrain model, and visual 
systems operation. 

Report D180-17875-1 , Flight Control and Procedures for Simulated Visual 
Approach and Landing - Self-Paced Training Package, describes the 
training materials develped for this study. Previous research in this 
program of the investigation of degradation of learned skills was 
covered in Report D1 80-1 5080-1 , Degradation of Learned Skills - A 
Review and Annotated Bibliography, Report D180-15081-1, Degradation 
of Learned Skills - Effectiveness of Practice Methods on Simulated 
Space Flight Skill Retention, and Report D1 80-1 5082-1 , Degradation of- 
Learned Skills - Effectiveness of Practice Methods on Visual Approach 
and Landing S{ii 11 Retention. 
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1. INTRODUCTION 

There has been considerable involvement in the evaluation and opti- 
mization of methods for maintaining or retraining skilled performance 
in recent years. In a series of studies for NASA which began in 1970, 
elements of pilot skill degration, as a function of time without prac- 
tice and as a function of time without practice and as a function of 
retraining methods, were evaluated. These elements involved both 
emergency procedures (combination of cognitive and discrete psycho- 
motor tasks) and flight control (primarily continuous psychomotor 
tasks). The results showed that both procedural and control skills 
deteriorated unacceptably after 1-4 months of inactivity (Sitterley 
and Berge, 1972). As expected, a fundamental difference in skill deg- 
radation was evidenced between procedural and continuous control tasks, 
with procedural tasks degrading unacceptably in a much shorter period 
of time. This difference was further highlighted by the finding that 
static rehearsal (review of manuals, checklists, photographs) coun- 
tered procedural degradation while dynamic warmup practice (practice 
on actual equipment) appeared necessary for the retention of control 
skills. 

In a following study, essentially the same types of retraining tech- 
niques were evaluated for a more complex and operationally oriented 
piloting task (Sitterley, Zaitzeff, and Berge, 1972). Improvements 
were made in the static rehearsal refresher training which enhanced 
trainee involvement in the tasks. In addition, a new retraining 
technique called dynamic rehearsal was used. It featured the contin- 
uous dynamic present;^ion of all pertinent visual and information 
elements of the tasks as they occurred in the simulated cockpit en- 
vironment, but without any direct control interaction on the part of 
the pilot. 

The results of this second study showed that static rehearsal signif- 

r K 

icantly reduced degradation but did not totai'ly reinstate performance 
on the flight control tasks without the addition of dynamic warmup 
practice. On the other hand, dynamic rehearsal prevented degradation 
for all categories of tasks. The primary difference between static 
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and dynamic rehearsal was the inclusion of a more complete representa- 
tion of the visual flight environment with dynamic rehearsal. From 
separate analyses of flight phases, Sitterley, et al (1972), found 
that the benefits of the dynamic rehearsal method were most strongly 
apparent for the highly visual (VFR) portions of the flight. Rela- 
tively small improvement was found over the static rehearsal method 
for the instrument flight phase. 

It Was postulated that the integration or coordination of far field 
perceptual cues, which were so well reii', forced by the dynamic rehear- 
sal method, was the critical element of the retraining. Further, 
Sitterley et al (1972) suggested that the primary skill retention 
problem was related to the maintenance of the visual /perceptual ele- 
ments of the flight control skills. Certainly, manual control per- 
formance did degrade; however, with highly experienced pilots, the 
basic skill of integration of discrete control elements into a smooth, 
coordinated response was more resistant to degradation. Consequently, 
it would appear appropriate to concentrate on enhancing the reinfor- 
cement of the understanding of the mission profile and flight operations 
in relation to the out-the-window visual environment and perceptual 
cues. 

The retraining method with the greatest enhancement potential and 
cost benefit is static rehearsal. Since the dynamic rehearsal method 
was so successful, the major potential for enhancement will primarily 
involve improvements in open-loop simulator techniques. While develop- 
ment and operation of such an open-loop trainer is less costly than 
a dynamic closed-loop device, it is still associated with significant 
weight, space, power, and cost penalties. On the other hand, the 
static rehearsal method is inherently less costly in terms of hardware, 
software, and operation. While the static rehearsal method failed 
to completely eliminate skill degradation, it did significantly reduce 
degradation from the no practice levels. The fact that it did so well 
as a retraining method, even in its relatively/ unrefined state, sug- 
gests that significant improvements in static retraining effectiveness 
are possible. 
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Improvements in static rehearsal retraining can be made in three prin- 
cipal areas. First, more pictorial information along the flight path 
should be included. In t‘"C’ Sitterley and Berge (X972) study, for example, 
only eight points in the flight were depicted. Three of them were under 
IFR conditions and did not require instrument and far field visual cue 
integration. More visual representations of altitude and line up before 
reaching the flare point were apparently necessary. Second, pictorial 
representations of off-nominal flight paths appear required to permit 
comparisons with the normal flight profile in order to give pilots the 
basis for recognizing poor performance. This very important approach was 
used for the dynamic rehearsal method. Third, more active pilot involve- 
ment with the static pictorial information is required to reinforce and 
strongly establish the critical perceptual cues of the visual environment. 
This involvement can be obtained by requiring specific responses from the 
pilot while he uses the static rehearsal training materials. 

If improvements in the content, format and use of static rehearsal methods 
are successful, the total retraining system costs can be reduced to a 
fraction of that required for dynamic display retraining methods. Fur- 
ther, costs of initial skill acquisi tion. and ground training may be re- 
duced by making more effective use of considerably, less flight simulator/ 
trainer time than currently employed. 


Purpose 

The purpose of this study, therefore, was: 1) do develop an advanced 
static retraining method which incorporated the recommended improve- 
ments in content, format, and use; .and 2) to evaluate the Improved - 
static retraining method by comparing .its effectiveness to the pre- 
viously investigated^methods under the. same. simulated flight conditions. 


3 
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2. METHOD 

Experienced pilots were trained to fly a simulated spacecraft of the H-33 
Space Shuttle orbiter configuration through an approach and landing. The 
simulation and experimental methodology was carefully controlled to dup- 
licate that of a previous study of practice method effectiveness (Sitter- 
ley, et al, 1972). Flight control data was measured at the end of train- 
ing and again at the end of 4 months for comparison to the data of the 
previous study. 


Subjects 

Five experienr^jji pilots were selected from Boeing engineering and ground 
school flight training staffs. As in the previous study, the subject 
population was required to meet the following criteria: (1) previous 

formal flight training and experience as a pilot; (2) commitment to no 
flight activities during the test period; (3) vision 20/30 corrected or 
better; and (4) under 55 years of age. 

The average age of the pilot population was 51.6 years with a range of 
50 to 54 years. The experience level of the subjects averaged 7,^80 
pilot hours with a range of 5,000 to 11,000 hours. They averaged 940 
instrument hours with a range of 800 to 1100 hours, The pilots averaged 
4.5 years since their last flight with a range of .5 to 9 years. 

Task Description 

The pilot's task was to control the vehicle from an altitude of 31,400 
feet through a descending turn to an approach and landing on a runway. 
Figure 1 depicts a schematic of the basic flight profile which required 
approximately 6 min, 45 sec. to complete. The mission description, 
approach data, and charts are described in the previous study report 
(Sitterley, et al , 1972) and in the self-paced training package used in 
this study (Helsel and Sitterley, 1973). Basically, the flight profile 
assumed that the pilot had just made a successful de-orbit and reentry 
pass through the transition" stage. 


fiOOOxSSCl li! l™ ) 

tew (s) 


Figure 1: Flight Sc hematic- Edwards AFB (SIM) Simulation 

Approach, H-33 Orbiter 

The test mission began 15 nautical miles from a simulated Edwards TACAN. 
The approach and landing were made unpowered. Ceiling was 10,000 feet* 
overcast, visibility 15 miles; the cloud deck was solid through 35,000 
feet. A turning approach descending at about 5,000 feet per minute 
was made to the TACAN using instruments only (IFR). Energy management 
was accomplished through judicious use of speed boards at an equivalent 
airspeed of 240 knots. Stabilization on the localizer and glideslope 
provided a straight-in approach to the Edwards runway 12 miles from the 
TACAN. During this portion of the flight, the pilot was required to 
perform emergency procedures to correct a series of malfunctions in the 
vehicle's flight control system (SAS; Failure Procedure). 

After crossing the TACAN station, a complete electrical power failure 
occurred which required the pilot to perform a corrective procedure 
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(Subsystem Scan). During the failure, the vehicle was repositioned 
to one of a standardized set of offsets from the flight path* These 
offsets, presented in random order, permitted the evaluation! of the 
final visual approach performance from a known starting point for all 
pilots. Upon power recovery (in 12 seconds), the pilot continued the 
descent on instruments through 10,000 ft, applying corrective control 
inputs to return the vehicle to the desired flight path. 

The pilot broke out visually at 10,000 ft and was able to use both the 
instruments and external visual environment to establish the required 
lineup and glideslope. At 8,000 ft, the on-board terminal navigation 
system failed, and the pilot was required to perform another corrective 
procedure. No correction of the failure was possible, forcing the pilot 
to make the remaining approach and final touchdown under visual conditions 
(VFR) with only basic vehicle attitude, speed, and altitude information. 

Equipment 

The experimental test was conducted using the visual flight simulation 
facilities of the Boeing Aerospace Company in Seattle, This equipment 
was the same as used in the previous study (Sitterley, et al , 1972) and 
Included the cockpit with associated displays and controls, the visual 
simulation system, and the computer and simulation control system. The 
equipment and associated computer software was integrated to provide a 
highly realistic simulation of a fully aerodynamic Space Shuttle orbiter 
descent, approach and landing as controlled visually and by instruments 
from a one-man cockpit. 

Cockpit 

A one-man cockpit, used for general purpose part- task simulation studies, 
configured with all displays and controls required to fly the simulated 
mission was used for both pilot training and retention testing. No 
attempt was made to duplicate any Space Shuttle cockpit concepts. Fig- 
ure 2 shows the general cockpit display/control configuration in rela- 
tion to a simulation pilot. 


6 


D180-17876-1 



Figure 2: Simulation Cockpit with Pilot 


The external out-the-window visu?! scenes were simulated using a 21-inch, 
1029 line T.V. monitor. An infinity optics system provided a field of 
view of approximately 40 degrees through the centrally located windscreen. 

The cockpit displays included electromechanical and cathode ray tube dis- 
plays for attitude, velocity, altitude, course, and status information. 

An X-20 type, two-axis, sidearm controller provided proportional rate 
commands for pitch and roll. Rudder pedals provided displacement com- 
mands for yaw. Pitch trim and speed board commands were provided through 
discrete rate controls. 
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Visual Slmul at ion System 

The external environnient seen hy the pilot through the electro-optical 
wind'^reen display was produced by high resolution television cameras 
which were con’puter controlled to "fly" over terrain models. The 1 inch 
vidicon cameras operated with a 1029 line standard. Each camera was 
mounted on a rail nuided carriage and ginbal system which provided 6 
degrees of freedom of motion. The computer controlled carriages and 
gimbals were digitally positioned over the two terrain models. Pre- 
cision control was maintained with both positional and velocit> feedback 
signals to an accuracy of 0.001 in. in translation and 3 minutes of arc 
in rotation. 



Figure 3: Terrain Model II - Edwards AFB, 

with Camera Stage and Lighting Mirror 
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Two scale relief terrain models were used during the visual portion of the 
approach. These models provided a realistic view of a modified approach 
to Edwards AFB frjm an altitude of 10,000 ft to touchdown. Figure 3 
depicts Model II of Edwards AFB and one camera/servo system carriage. 

The model was 11 ft x 24 ft (Scale 1:6250) and provided terrain feature 
representation to a vehicle altitude of 175 ft. Figure 4 depicts the 
camera eye view of Model II, approxirately 5 miles from the runway 
threshold. 
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Model I ^rovided the detailed representation of the runway for pilot's 
eye altitudes of 300 ft to 20 ft (Scale 1:200). The 11 ft by 90 ft 
H'odel is depicted in Fiqure 5 alonq with its cainera/servo system. During 
pilot trainino anti perfoniiancc tcstinn, the Model I approach lights 
and adjacent terrain were replaced with dry lake bed features, scaled 
and contrast matched to Model II . 



Figure 5: Terrain Model I - Runday, and Camera/Servo System 


The visual transition between the two models occurred when the vehicle 
passed through an altitude of 300 ft. The landing model camera stage 
was synchronized with the vehicle's flight while the other camera stage 
was still flying. Visual transition was accomp'' ished by computer con- 
trolled video fade-in/fade-out of the two TV camera/terrain model systems. 
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Computer System 

Simulation of the flight vehicle aerodynamics, flight control and cockpit 
information display, and visual simulation system control were accom- 
plished using an XDS 930 digital computer operated in conjunction with a 
Varian 662i digital computer, a Sanders ADDS 900 graphics display system, 
and analog to digital and digital to analog conversion equipment. The 
mathematical model which described the dynamic flight of the H-33 orbiter 
vehicle and the flight environment was programmed for real time solution 
on the main digital computer. The model provided a relatively sophis- 
ticated description of the vehicle including aerodynamics forces and 
moments, dynamic pressures, and the flight control system, as well as 
the flight environment in terms of wind accelerations, velocities, shear, 
and gusts. Figure 6 depicts the general characteristics of the H-33 
vehicle. 

Input commands from the pilot in the cockpit and programmed environmental 
conditions were used to compute the vehicle attitude, position and ve- 
locity information. This information was sent as operation commands to 
each axis of the servo system which oriented the high resolution TV 
cameras over the scaled terrain models. The resulting video signal was 
then processed and fed to the large high resolution TV display in the 
cockpit. Simultaneously, vehicle attitude, position, and movement data 
was processed for display on the cockpit instruments. 

Throughout each simulation flight, the specified flight performance 
data was collected and stored. At the end of each flight, the .32 
flight performance measures were printed along with pilot's names, 
session and flight numbers, and corresponding experimental conditions. 
After each set of five flights, block summary data and standard devi- 
ations were printed. 


11 


D180-17876-1 



12 




















D180-17876-1 


Procedure 

The experiraental procedure was identical to that pi-eviously used {Sitter- 
ley, et al, 1972) and was broken down into three general phases. The 
first phase included the initial briefing and training of the test sub- 
jects to perform the flight control and procedure tasks. This phase 
concluded with qualification testing Of the pilots. The second phase was 
the 4 month retention interval. During this 4-month period of the sub- 
ject's absence from the simulated space mission and normal flying, the 
advanced static retraining materials were developed. The third phase 
involved retraining the subjects using the new self-paced training pack- 
age and carrying out retention testing of the subjects. 

Pilot Training 

All pilot test subjects completed flight and procedure training in the 
same manner as in the previous study. The five pilots were introduced 
to the task and provided with copies of the original flight control and . 
procedure manual. They then attended a ground school briefing and cock- 
pit familiarization. Following the ground school, the pilots received 
IFR and visual flight and landing practice, procedure task training, 
and full mission flights with emergency procedures. 

All pilots were trained and qualified on both the flight control and 
procedure tasks according to the criterion previously used; essentially 
asymptotic performance levels. The desired goal was for touchdowns 
2000 feet down the runway with sink rates of 4 to 6 feet per second, on 
centerline, gear down, and yaw and bank angles near zero. Unsatisfactory 
performance was defined by any of the following: sink rates greater than 
12 feet per second, touchdown short or wide of the runway, landing gear 
up, and yaw or bank angles greater than 10 degrees at touchdown. 

The flight control tasks required an average of 60.8 landings per pilot 
to train to proficiency, with a range cf 51-75 landings. In terms of 
simulator training time, the pilots required an average of 5 hours at 
the controls to reach qualification. The pilots completed slightly more 
part mission flights {VFR only) than previously used, which resulted in 
more landing practice in slightly less total flight time. 

13 
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An average of 140 procedure task trials were required for each pilot to 
reach qualification {86 to 197). The time expended for ground school 
briefing and procedure task training averaged 3.1 hours per pilot (as 
compared to 2.8 hours in the previous study). The average total train- 
ing time per pilot amounted to 8.1 hours over 5.4 sessions (as compared 
to 8.8 hours over 5.3 sessions in the previous study). 

Upon completion of training and collection of the training qualification 
performance test data, all trafning materials were recovered from the 
pilots. The pilots were informed that they were entering the 4 month 
retention interval phase. During the retention interval, they were not 
to return to the simulator laboratory, discuss the simulated flight, or 
perform any piloting functions in other flight simulators or actual air- 
craft. The pilots were told they would be contacted regarding their re- 
tention test schedule two weeks before the end of the retention interval. 

Training Data Analysis 

This study was designed to evaluate the efficiency of an improved static 
retraining method and compare it with the methods previously investigated 
(Sitterley, et al, 1972). During the course of the previous study, sig- 
nificant, individual differences in basic flight skills were detected be- 
tween the subjects. In order to equate initial performance between the 
original retraining method groups, the subjects had been assigned to 
groups by skill level using a matched groups design. Th4“ five pilots in 
the present study likewise demonstrated noticeable differences in perfor- 
mance. However, it was no longer., possible to use a matched groups design 
as subject assignment to the original retraining method groups was al- 
ready established. 

In this study, the training qualification performance of the pilots 
assigned to the new method group (Group IT: Improved Static Retraining) 
was com, pared with the performance of the three previous groups (Group I; 
No Practice, Group II; Static Rehearsal, Group III: Dynamic Display) 
using a subjects nested within groups analysis of variance design. A 
total of 32 ANOVA's were performed, one for each of the flight control 
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measures. These analyses used the absolute error data for flight control 
performance. As procedure task performance was not of interest in this 
study, no procedure task data was analyzed. 

The probability of significant differences between-groups and subjects 
nested within groups is depicted in Table 1 for each of the 32 perfor- 
mance measures obtained during pilot training. In addition, the average 
performance of the current Group IV subjects for each measure is listed 
as well as the grand mean for the first study in Groups I to III) 
and grand mean for the current study in Groups I to IV). Inspection 
of the performance means indicated that the new subjects achieved a high 
degree of proficiency at the end of training and performed comparably 
to those trained in the previous study. 

The AMOVA results in Table 1 show that no significant differences were 
detected between the four treatment groups for any performance measure. 
While the null hypothesis that no differences exist cannot be proved, 
inspection of the data and, the. absence. of detectable differences suggests 
that the current subjects were suitably and. comparably trained. As with 
the previous. study, significant subjects within groups differences were 
detected. for most of the flight performance measures. 

Improved Static Retraining Method Development 

The improved static retraining method. was developed to meet the concept 
of a self-paced retraining package which could be used effectively with 
little or ho supporting training equipment. .Development of the retrain- 
ing package.was accomplished in six iterative phases: 1) definition 

of requirements and flight phases;. 2). specification of behavioral ob- 
jectives; 3) story boarding of task requirements and sequence; 4) 
definition of detailed. task requirements, 5) formatting and layout 
of training. materials; and .6). production of the training package. 

Well defined phases which. the pilot. could use to continuously visualize 
his present position with respect to the runway were. used to provide a 
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TABLE 1; 


Pilot Performance and Analysis of Variance 
Results (F Ratio) at Completion of Training 


performance measure 

NEW 

GROUP 

MEAN 

GRAND MEAN 

ANOVA SOURCE 1 

FIRST 

STUDY 

current 

STUDY 

GROUPS 

SS WITHIN GROUPS 
CROUPS 


1 

ALTITUDE ERROR IFTI 

336 

357 

327 

0.676 

3.673* •• 


2 

LATERAL ERROR IFTI 

627 

666 

631 

0.416 

3,667* •• 

< 

3 

HEADING ERROR lOEGI 

2£ 

1^ 

1J 

2.053 

1.670 


4 

VELOCITY (KTsI 

243 

346 

246 

0.994 

1,471 

H 

G 

DESCENT RATE ERROR IFT/SEC) 

10,6 

13.0 

11.7 

CAM 

2.041** 


6 

INTEGRATED VELOCITY ERROR IKT-SEC) 

1002 

971 

978 

0794 

6.922*** 


7 

ALTITUDE ERROR IFTI 

375 

233 

336 

0.464 

3.218*** 


B 

LATERAL ERROR IFTI 

t£ 

100 

96 

0.666 

4.766* 

ui 

9 

heading ERROR IDEGI 

1.1 

1.2 

U 

0.336 

1.191 

c 

10 

velocity IKTSl 

330 

230 

330 

0.068 

3.448*** 

5 

U 

DESCENT RATE ERROR (FT/SECI 

'14J 

13.7 

13J9 

0.188 

6296*** 


13 

integrated velocity error IKT-SECI 

1005 

•06 

186 

1J34 

1822** 


13 

integrated ALTITUDE ERROR IFT-SCC) 

37,171 

36>M 

38.601 

0J04 

1246 


14 

integrated lateral ERROR IFT-5ECI 

73,662 

70,367 

71,171 

0.120 

0.726 ' ' 


1G 

ALTITUDE ERROR IFTI 

64 

46 

60 

0.661 

3801 ••• 

o 

IE 

LATERAL ERROR IFTI 

24 

36 

26 

0.297 

3,774* •• 

o 

17 

heading error idec) 

0.6 

0.7 

0.7 

0.188 

2.351 ••• 


le 

VELOCITY IKTS) 

192 

199 

190 

QB41 

3211*** 

22 

19 

DESCENT rate ERROR IFT/SECl 

11.0 

•J 

9.4 

0,318 

2277*** 

X 

20 

INTEGRATED VELOCITY ERROR (KT-SECI 

667 

607 

697 

0.647 

4.162*** 


21 

integrated ALTITUDE ERROR IFT-SECl 

4124 

3332 

3456 

0.631 

3224*** 


22 

integrated lateral ERROR IFT-SECl 

10^ 

1297 

1240 

0.368 

3JI19*** 


23 

Lateral error iftI 



21 

33 

0,210 

3,992* •• 


34 

DOWN range error IFT) 

1126 

•46 

•90 

0.728 

2.173** 


26 

heading error IDEGl 

0.6 

0.6 

0.6 

0.194 

1.097 


26 

VELOCITY IKTSl 

167 

163 

160 

1.296 

7.462* •• 

s 

27 

DESCENT RATE (FT/SECi 

S.2 

7.2 

7,6 

0,763 

1.663* 


28 

bank ANGLE IDEGI 

1.0 

0.8 

0,9 

14)63 

1J21*» 


29 

PITCH ANGLE (DEGI 

11.4 

10.8 

10J 

0833 

10,386*** 


30 

INTEGRATED VELOCITY ERROR {KT-SECI 

672 

660 

688 

0JS76 

6.470*** 


31 

integrated altitude error IFT-SECl 

392 

236 

276 

1.334 

3£84*** 

_ 

33 

integrated lateral error IFT-SEC) 

3B3 

196 

217 

1.081 

2i90*** 


•p < 0,10 

**D < 0,05 

•••p < 0.01 


meaningful framework for the retraining task. The first step in develop- 
ing the self-paced training package was, therefore, to divide the flight 
profile into flight phases with definite beginning and end points. The 
flight phases were selected consistent with a constant set or family of 
control actions (Figure 7). Beginning and end points for each phase were 
selected which permitted rapid recognition of obvious instrumentation 
information status or change. 
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INITIAL 

POSITION 



ALTS 31,400' 
SPD: 240 KT 
HDG.; 225“ 
RADIAL:; 135“ 


FLAR5 

THRESHOLD 

touchdown 


SAS FAILURES OCCUR 
DURING TURN-IN 


AUX POWER FAIL 
MAY OCCUR 12-15 SEC 
JUST AFTER CROSSING 
TACAN 


OFFSET MAY 
OCCURS 5 SEC 
AFTER AUX POWER 
RETURNS 


TERM NAV 
ILS LOCALIZER 
GLIDESLOPE (GS) 
AND HEADING 
CAN FAIL AT 
9000' 


Figure 7; Flight Profile Broken into Flight Phases 

After the basic flight phases were established, preliminary specific be- 
havioral objectives were defined foi^each phase. These objectives stated: 
1) what the pilot had to do, 2) the specific conditions under wliich he 
was expected to do it, and 3) satisfactory performance criteria." These 
objectives were updated later as a result of t'ie detailed task require- 
ments definition. They served as the basis for defining the basic train- 
ing requirements and the identification of the critical visual cues. An 
overlying purpose of the specific ,behavi oral objectives in final form was 
to convey to the pilots exactly what was expected of them during each 
phase and to provide them with a self-test covering' the mission require- 
ments and performance. 

The'flight profile and preliminary specific behavioral objectives were 
used to define critical information cues and cockpit photo requirements. 
These requirements included the specific photos for each phase as well as 
photo frequency (every 5 to 15 sec) required to capture the changing in- 
strument environment cues needed by the pilot. A series of cockpit photos 
were then taken of a near perfect and selected off nominal flights to 
provide adequate coverage of situations to which the pilots were expected 
to successfully respond. These photos were storyboarded and analyzed to 
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see If they met the content requirements of the specific behavioral 
objectives (Figure 8) 


Figure 8: Cockpit Photo Storyboard of Flight Phases for 

Off-Nominal Flights 

Using the storyboard as a guide for the definition of the detailed task 
requirements, the critical information and instruments were then identi- 
fied with phase of flight. A detailed breakdown of all primary and sec- 
ondary information requirements was prepared by instrument and flight 
phase. The rpsulting matrix provided immediate visibility of all display 
and control elements to be covered in the retraining. Using this matrix 
as a starting point, iterative analyses of the tasks in each phase were 
made to identify: 1) input information requirements, 2) pilot actions, 

and 3) outputs for significant and critical tasks. As a result of this 
ongoing analysis process, the definitions of the flight phases were up- 
dated . 
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The analyzed pilot training data was used In the training requirements 
definition process to provide useful Information In further Identifying 
flight control problem areas. Some of these problems were also identi- 
fied from the results of the previous study. The major problem areas arc 
summarized as follows; 

A. Forgetting the basic cockpit layout and 
operational nomenclature of display symbology. 

B. Forgetting the flight phase sequences and the 
corresponding correct speed, heading, attitude, 
etc., for each phase.! i 

C. Forgetting correct display patterns for specific 
roll angle and flight path off-set conditions. 

D. Forgetting the cues used after transition 

from IFR to VFR for final approach and touchdown. 

Based upon the task requirements analysis and specification of critical 
problem areas, the specific behavioral objectives were updated. Addi- 
tional cockpit photographs were taken where required to support the 
training process and a graphical representation of important display 
symbology was prepared to reinforce recognition of off-nominal flight 
path conditions. The general mission requirements and associated speci- 
fic piloting task requirements for flying the spacecraft from the initial 
position to touchdown provided the "design to" goal for the retraining 
package. Each portion of the retraining had to contribute to one or 
more of the task objectives. Repetition and variety was used to em- 
phasize key information and retain student interest. 

Once the retraining requirements were established, the general format 
of the training package was established in book form. The book format 
was used to readily organize the illustrations and text of the training 
package. The overall philosophy was to move from the general to the 

.*5 

specific: first, to re-famili arize the pilots v;ith the purpose, general 
vehicle characteristics, and the flight missions; and second, to pro- 
vide detailed cockpit familiarization, emergency procedures and flight 
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training. The package was designed so that the pilots could repeat 
those portions of the course in which they were weak. 

The self- test was prepared in printed* worksheet format based upon the 
training objectives. These worksheets were reproductions of various 
pages of the training package with specific information left blank. 

The purpose of the self-test was to further reinforce the important 
information required to successfully fly the mission through the iden- 
tification of significant flight parameters, display symbology, control 
actions, and visual cues. Responses were to be recorded on the work- 
sheets from memory if possible, and then completed and verified with 
reference to the training package. 

The complete training package was produced in a printed booklet in 
order to provide review flexibility. In addition, a sound/slide pre- 
sentation of a complete flight was prepared using the same photos con- 
tained in the printed training package. The slides were presented as 
a continuous sequence of "stop-action" frames synchronized to the flight 
time base. The objective of the sound/slide presentation was to provide 
a final summary review which included time and rate cues for mission 
events. 

Self-Paced Training Package Description 

The self-paced training materials consisted of a printed training pack- 
age in 11 X 17 inch booklet form; 35 mm black and white, and color 
slides with synchronized audio recording; and a pictorial self-test. 

The following is a summary of the printed training package, "Flight 
Control and Procedures for Simulated Visual Approach and Landing - 
Self-Paced Training Package" (Helsel and Sttterley, 1974). The table 
of contents for the training package is depicted in Figure 9. The 
"Introduction" section of the course provided the pilots with the ob- 
jectives for completion of the flight, a description of the H-33 vehicle, 
and what the desired flight profile was." '0ther training objectives 
were provided to help focus the pilot's attention on what was important 
and what he was expected to do after he had completed the self-paced 
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I. Introduction 7 

A. Objectives 9 

B. H-33 Space Shuttle general characteristics 10 

C. Flight profile summary 12 

II. Cockpit familiarization 15 

A. Identification of functional groups 16 

B. Instrument group location/description 18 

C. Control group location/dcscription 26 

D. Status panel group location/dcscription 28 

III. Emergency procedures training 31 

A. Stability augmentation system (SAS) 32 

B. Major subsystems 34 

IV. Flight training _ 37 

A. Flight profile summary 38 

B. Near perfect flight (all phases) 41 

C. Off flight path recognition 58 

D. High-left offset sequence 61 

E. 8,800 ft. through 800 ft. sequence 67 

F. Flare through touchdown sequence 73 

V. Summary 81 


Figure 9: Self-Paced Training Package Table of Contents 



package. Selected pages from the training package itself, with call outs 
and specific information missing, were also provided to the pilots as a 
self- test. These test/work sheets served to correct the student and 
reinforce his learning in preparation for entering the simulator for the 
skill retention test. 


The H-33 Space Shuttle general characteristics portion of the "Introduc- 
tion" contained a line drawing of the space shuttle with overall dimen- 
sions and weights along with a brief description of the vehicle's char- 
acteristics. The flight profile summary contained a plan view, eleva- 
tion view, and a three-dimensional perspective view of the flight path. 
Initial position (IP), touchdown and seven (7) phases were defined as 
well as the location of the TACAN, ILS and runway. The flight path was 
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covered in considerable detail to provide a framework around which the 
pilot could fill in the required detail as he proceeded through the 
training package. 

The "Cockpit Familiarization" section appeared next to reacquaint the 
pilots with the cockpit and particularly with the Electronic Attitude 
Director Indicator (EADI), Multifunction Display (MFD), and the status 
panels. While the function, range, and units of each instrument was 
reviewed, the EADI and MFD displays were covered in greater detail be- 
cause they were unique. The pilots had to remember what the display 
symbols indicated from their shape and white/grey/black shade. A series 
of recognition patterns was provided for efficient identification of 
instruments, controls and status. 

"Emergency Procedures" for the stability augmentation system (SAS) and 
major subsystems were discussed next. The emergency procedures were at 
this point in the sequence so that Vihen the near perfect flight sequence 
was covered, the pilot could imagine a SAS or major subsystem failure 
and mentally trace through the emergency procedure. 

"Flight Training" started with a review of a three-dimensional perspec- 
tive view of the flight profile. The profile contained all important 
altitude, attitude, position, and velocity information for the initial 
position, and the beginning and end of each of the seven flight phases 
ending with touchdown. An abbreviated copy of this three-dimensional 
perspective flight profile was used in each phase of the near perfect 
flight sequence to help the student visualize where he was as he re- 
viewed the detail on cockpit instrument panels. This flight profile 
was repeated for each phase in order to reinforce the flight phase 
sequence . 

The near perfect flight sequence consisted of cockpit photographs at the 
beginning and end of each flight phase. The instruments that the pilot 
should observe and what he should do using the controls was listed be- 
tween these photographs. In addition, special notes were supplied which 
described in more detail what the student should look for in terms of 
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instrument and visual out-the-window cues. This format allowed the stu- 
dent to see what the cockpit instruments looked like at the beginning and 
end of each phase, and what he had to observe and do. Next, a series of 
display symbology recognition patterns for all off-nominal flight path 
conditions was provided for review. The basic corrective maneuver was 
also included with each graphical recognition pattern, A series of cock- 
pit photographs which depicted the recovery from a High/left off-set 
flight path condition was provided following the pattern recognition re- 
view. This photographic sequence permitted the pilot to visualize the 
important display and visual information associated'with the most complex 
corrective maneuver. 

The flight training section concluded with two photographic flight se- 
quences. The first, from 8,800 ft to 800 ft, was designed to allow the 
pilot to visualize the transition from IFR to VFR and to establish the 
proper mental image of the out-the-window scene. These photographs were 
taken at 15 second intervals, showing the proper visual image of line-up, 
altitude, and aim point, to help the pilot arrive at the flare point at 
the correct distance from the runway. The last sequence was from flare 
(750 ft) to touchdown. This sequence was photographed at 5 co 12 second 
intervals to establish a good out-the-window image and instrument/out-the- 
window eye scan pattern. p. 

The "Summary" section directed the pilot to review several critical por- 
tions of the training package a second time. Additional key portions 
of the training package were listed with recommendations for review if 
the pilot was unsure of his understanding/. The summary concluded witn 
the automatic stop action sequence^^of 33 cockpit photographs of the 
entire flight. This slide/sound sequence was presented at a rate which 
approximated real time. 

Retraining and Retention Testing 

One week prior to the date the first subject was to be retested, a com- 
plete checkout and recalibration of all simulation equipment was accom- 
plished. Cockpit flight control output voltages previously recorded during 
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the training phase were reflown through the computer prior to testing to 
determine the empirical equivalence of the flight profile, display control 
operations, and the visual scene camera servo system. Comparisons of the 
flight performance data recorded during training and the data obtained 
prior to retention testing as well as the subjective testing flights flown 
by the experimenter pilots indicated that the simulator was recalibrated 
to the condition that existed during the training and qualification test- 
ing. In addition, the experimenters and laboratory personnel practiced 
all test of>rat1ons to ensure that the experimental procedures were con- 
sistent with those previously used during training and were performed 
Without error. 

Each pilot was scheduled to complete retraining using the new self-paced 
materials immediately prior to the retention test at the end of the 4 
month interval. Only one pilot exceeded the prescribed interval limits 
(120 days, - 3 days) and was retested after 129 days. 

Immediately prior to returning to the laboratory for formal retraining 
and retention testing, each pilot was provided with a copy of the self- 
paced retraining materials for review purposes. This review was self- 
controlled by the pilots who reported spending an average of 2 hours going 
over the material. On the day scheduled for retesting, each pilot comple- 
ted a formal 40 minute review in the briefing room. 

This review was self-administered and consisted of: 1) an "open book" 

self-test covering the important flight control information and require- 
ments, (2) the sel-^-paced training package and photographic flight sum- 
mary review, and (3) a pictorial 35 mm slide/sound flight profile/opera- 
tions review. The briefing room was arranged to provide ready access 
to all of the retraining materials (Figure 10). In addition, a complete 
photographic sequence of the cockpit during the flight was available for 
review on the briefing room wall. 

The series of 39 slides, depicting the cockpit and out-the-window visual 
environment, was^rear screen projected at full size (Figure 11). Each 
slide was automatically advanced following the real time sequence of 
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Figure 10: Pilot in Rriefing Room Reviewinn Self-Paced 

Retrainino Material? 



Figure 11: Pilot Watching Rear Projected Cockpit Photograph 

Sequence of Complete Flight 
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mission events. The recorded narration, which emphasized critical cues 
and control events, was automatically synchronized with the slides. 

After completion of the 40 minute review, the pilot returned to the simu- 
lator area and was read a standard set of instructions covering seat and 
rudder pedal adjustment and simulator operation. The pilot was then allow- 
ed a few minutes to refamiliarize himself with the cockpit, the instruments, 
and control locations. 

The first retention test flight was then started. The data from this flight 
was used to assess the effect of the advanced static retraining on retention 
of flight control skills. After completion of the retention -test flight, 
the pilot flew four additional flights. Data was collected on all flights 
and at the end of each flight the only feedback information that the pilot 
received was descent rate and distance down the runway at touchdown. 

The four additional flights in combination with the retention test flight 
provided a total of five hands-on practice flights. During these flights, 
the pilot could become more familiar with the dynamics of the vehicle opera- / 
tion, instrumentation and visual cues. Upon completion of the flights, the / 
pilot w^s allowed, a 10 minute break. I 

At the end of the rest period, the pilot was once again tested on his ability 
to successfully fly the simulated approach and landing mission. The data 
from this sixth retention test flight was used to assess the effectiveness 
of a combination of the advanced, self-paced retraining method and hands-on 
practice. 
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3.0 RESULTS 

Eleven performance measures were used to evaluate flight control perfor- 
mance. Six of the measures were repeated at four points in the flight, 
two measures were taken during three flight phases, and two additional 
measures were taken at touchdov/n. This provided a total of 32 flight 
control data measurements which were identical to those used in the 
previous study (Sitterley, et al., 1972). 

Based upon three of these measures, one critical measure of operational 
significance was derived; landing success; that is, did the vehicle land 
safely on the runway with a descent rate within the tolerance of the land- 
ing gear structural strength. In addition, 26 of the 32 individual per- 
formance variables were integrated in a combined flight performance meas- 
ure to assist in the overall interpretation of the results. 

Crash Landing Criteria 

r. 

In the previous study it was reported that the absence of any type of 
retention interval practice was disastrous. Each. of the five pilots in 
Method Group! crash landed the vehicle at the end of the four month re- 
tention interval as defined by one or more of the. crash condition criteria 
(long/short, wide, or Bard). Dynamic warmup practice afforded by the five 
practice. flights reduced the number of crash landings to two. Static re- 
hearsal practice (Group II) also resulted in only two crash landings. 

The addition of warmup practice. to static rehearsal practice eliminated 
the incidence of crash landings completely. In the previous study, 
only the dynamic. rehearsal (Group. Ill) -resulted in no crash landings at 
the end. of. the retention interval . 

t'- n 

In the. current study, no crash landings occurred using- the improved static 
retraining (Group..IV). In.terms.of this practical measure of successful 
performance, the. improved. static retraining-method. equalled the effective- 
ness of the dynamic rehearsal method. of the previous study. 
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Combined Flight Performance Measure (CFPM ) 

The CFPM is an e)<pression of overall piloting performance throughout 
the entire flight in one measure. The measure was determined by equally 
weighting all of the error performance measures (except heading) at each 
of the four critical flight control points (TACAN, Flare, Threshold, 
Touchdown). Heading "errors" (deviations from an ideal course line) were 
not considered because these were generally less than 2 degrees and were 
usually indicative of a corrective action taken to decrease the apparent 
lateral error at the moment. Likewise, pitch and bank angle error at 
touchdown were not included in the combined measure as the deviations 
were very small and usually corrective in nature. Thus, oP the 32 flight 
performance measures, 26 were used to derive the CFPM. 

A baseline performance level was determined for each measure by its av- 
erage value in all qualification performance tests. This nominal or 
"quaT’ level was used to establish the performance factor or ratio for 
each data measurement that was taken. That is, the flight performance 
factor for a data measurement was the actual value measured, divided by 
the mean of that parameter in all qualification tests of both the previous 
and the current study. Since the CFPM was evolved to give a picture of 
the flight overall, all the parameters were given equal weight. The 26 
flight performance factors were, therefore, arithmetically averaged to 
provide the overall combined flight performance measure for each flight. 

Overall flight control performance was evaluated using the CFPM for the 
total flight. Figure 12 depicts the effectiveness of the improved static 
retraining method (Group IV) in comparison with the methods previously 
evaluated. As can be seen, the improved static method eliminated vir- 
tually all skill degradation. Furthermore, the data show that perfor- 
mance after using the improved method was better than any method previously 
used. 

- I 

II 

A tests by methods analysis of variance statistic (ANOVA) with subjects 
nested within methods was used to analyze differences between groups and 
performance tests on the combined flight performance measure. The retention 


28 


PERCENT SKILL DEGRADATION 


D180-17876-1 



100 


QUAL 

LEVEL 


\\ 


GROUP 1: 




K r- 



TRAINING ^ 

INACTIVITY ^ 

ADDITIONAL PRACTICE ^ 



(4 MONTHS) 

(4 FLIGHTS) 


QUALIFICATION RETENTION WARMUP 

TEST TEST TEST 


FIGURE 12: Skill Retention with Improved Static Retraining 
^Group IV) Compared with Previous Methods 
(Groups I, II, and III). (Based or Combined 
Flight Performance Measure.) 
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test data evaluated the effects of no practice, static rehearsal and 
dynamic display rehearsal for Method Group I, II, and III of the previous 
study and improved static rehearsal for Method Group IV in the current 
study. The warmup test data evaluated the effects of the addition of 
dynamic warmup practice to the training methods used by the four groups. 

t 

The results of this analysis are depicted in Table 2 for the overall 
flight, with significant differences (p c .01) detected for both main 
effects and for the interaction. The significant methods effect indica- 
ted that retention performance improved as a function of practice method. 
The significant tests effect and the methods by tests interaction showed 
that the benefit. .of warmup practice was most strongly associated with the 
groups that had less efficient or no retention training. 


TABLE 2: Analysis of Variance Results (F Ratio) for 

the Combined Flight Performance Measure 


MISSION PHASE 

SOURCE 1 

TESTS 

METHODS 

Tx M 

OVERALL FLIGHT 

27.26*** 

6.18*** 

7.89*** 

TAG AN 

10.69*** 

3.81** 

2.56** 

FLARE 

14.00*** 

4.08** 

4.53*** 

THRESHOLD 

4.39** 

3.42** 

1.77 

TOUCHDOWN 

1.63 

4 40** 

1.35 


;! *p < .10 

' **p < .05 

***p < .01 


The data were further analyzed using the Duncan *"s New Multiple Range 
Test. As previously reported (Sitterley, et al , 1972), performance of 
both the no practice group and the static rehearsal group was signifi- 
cantly degraded at the end of the retention interval while the dynamic 
display rehearsal group showed no significant and litt-le'-practical deg^ 
radation. The static rehearsal group performance Was significantly 
better than the no practice group and the addition of 'dynamic warmup 
practice significantly reduced the amount of degradation for Groups I 
and II . 
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Most importantly, however, was that no degradation was found for the im- 
proved static method (Group IV). In terms of the CFPM for overall flight, 
the improved static method resulted in better performance than the methods 
previously used. 

Similar results were obtained when performance during each flight phase 
was evaluated. The four flight phases were; 1) Start to TACAN (IFR); 

2) TACAN to flare (VFR); 3) flare to threshold (VFR)j and 4) thres- 
hold to touchdown (VFR). 

Figure 13 depicts performance as measured by the CFPM as a function of 
flight phase. The CFPM data for e^ch flight phase were analyzed using 
the same ANOVA as for the overall flight; these results are also shown 
in Table 2. As with the overall flight CFPM, the use of the improved 
static method eliminated skill degradation for all flight phases.- It is 
important to note that for the IFR phase (TACAN) the improved method was 
considerably more effective than the dynamic display method. As can be 
seen, the CFPM showed the same overall results in terms of method selec- 
tion as did the frequency of crash landings: improved static retraining 
was superior to the previous methods. 

Individual Flight Control Performance Measures 

Two basic analyses of each of the flight control data measurements were 
also performed. The first analysis compared performance at the end of 
training (qualification test) with performance using the improved static 
retraining method at the end of the 4 month interval (retention test) and 
after five hands-on practice flights (warmup test). The second analysis 
compared performance on the three tests using the improved static method 
with the performance data collected in the previous study. For both 
analyses, the analysis of variance (ANOVA) statistic was used to evalu- 
ate the results obtained from each of the 32 performance measures. 

The results of the first analysis, assessing the effectiveness of the 
improved static method for countering skill degradation, is depicted in 
Table 3. Included .in the table is the mean performance achieved by the 
five subjects on the three tests for each performance measure and the 
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FIGURE 13: Skill Retention with Itnproved Static Retraining. > 

(Group IV) Compared with Previous Methods 
(Groups I, II, and III) for Four Critical li 

Flight Phases. (Based on Combined Flight 
Performance Measure.) 
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TABLE 3: Improved Static Method-Pilot Performance and 

Analysis of Variance Results for Skill Retention 
Tests 


PERFORMANCE MEASURE 

PERFORMANCE TEST MEANS 

anova 

DUAL 

RETENTION 

WARMUP 

£.RAT(0 

P_ 


1 

ALTITLDE ERROR (FT) 

144 

227 

150 

0,556 



2 

LATERAL ERROR (FT) 

310 

205 

271 

0.481 


< 

3 

HEADING ERROR (DEG) 

1.6 

1.5 

1.8 

0.081 


o 

< 

4 

VELOCITY IKTS) 

243 

240 

242 

0,397 



5 

DESCENT RATE ERROR IFT/SEC) 

4 

6 

6 

0.415 



6 

/VELOCITY ERROR IKT-SECi 

714 

1498 

810 

3.404 

<10 


7 

ALTITUOE ERROR (FT) 

178 

160 

266 

0.558 



8 

LATERAL ERROR (FT) 

51 

57 

141 

3.198 

<10 


g 

HEADING ERROR (DEG) 

1.6 

2.0 

1.8 

0.095 


o: 

10 

VELOCITY (KTS) 

234 

237 

230 

1.101 


3 

11 

DESCENT RATE ERROR IFT/SEC) 

21 

e 

10 

2.456 


IL 

12 

/velocity error (KT-SEC) 

069 

940 

893 

0.045 



13 

/ALTITUDE ERROR (FT^SEC) 

29,SK 

50.BK 

39.1 K 

2,296 



M 

/LATERAL error (FT-SEC) 

116.7K 

159.5K 

106.7K 

4.396 

<10 


IS 

ALTITUDE ERROR (FT) 

36 

22 

67 

2,853 



16 

LATERAL ERROR (FT) 

32 

56 

60 

0.837 


3 

o 

17 

HEADING ERROR (DEG) 

,7 

1.3 

1.0 

0.652 


X 

18 

VELOCITY (KTS) 

192 

166 

186 

0.387 


uu 

19 

DESCENT RATE ERROR (FT/SEC) 

9 

5 

9 

0.604 


X 

20 

/VELOCITY ERROR (KT-SEC) 

564 

520 

664 



K 

21 

/ALTITUDE ERROR (FT-SEC) 

3037 

1760 

2478 




22 

/ LATERAL ERROFl (FT-SEC) 

1096 

858 

2093 

3.142 

<10 


23 

LATERAL ERROR (FT) 

37 

21 

18 

2.029 



24 

DOWN range ERROR (FT) 

871 

919 

969 

0,027 


z 

25 

HEADING ERROR (DEG) 

.6 

1.8 

1.5 

1.103 


1 

26 

VELOCITY (KTS) 

160 

187 

167 

12.806 

<01 

Q 

27 

DESCENT RATE (FT/SEC) 

10 

9 

9 

0,206 


z 

u 

28 

BANK angle (DEG) 

0.8 

1.8 

1 .6 

1.316 


D 

O 

29 

PITCH ANGLE (DEG) 

11 

8 

11 

6.166 

<,05 

p 

30 

/VELOCITY ERROR (KT-SEC) 

639 

218 

615 

6.940 

<.0E 


31 

/altitude ERROR (FT-SEC) 

280 

332 

275 

0.064 



32 

/lateral ERROR (FT-SEC) 

277 

136 

269 

0.726 



•P < .10 
••P < .Ob 
'••P<.10 

associated results of the repeated measures ANOVA. As can be seen by in- 
spection of the data in Table 3, little practical or significant degrada- 
tion occurred in performance when the impvoved static retraining method 
was used. 
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Of the seven measures for which strong trends or significant test dif- 
ferences were detected, only four were associated with degradation on the 
retention test {Measures 6, 14, 26 and 29). Of these four, the lower 
pitch angle (measure 29) on the retention test was complemented by the 
higher touchdown velocity (measure 26) which resulted in a satisfactory 
descent rate (measure 27). 

Each of these performance measures were also subjected to the analysis of 
variance statistic to evaluate the effectiveness of the type of refresher 
training on skill degradation. A two factor (retraining methods by per- 
formance tests) experimental design with repeated measures on the test 
factor (subjects nested within groups) was used for this second analysis. 

When compared with the retraining method.s used in the previous study, the 
improved static method resulted in very high performance. Table 4 depicts 
the mean performance for each measure as a function of practice methods 
used in the previous study (Groups I, II, and III) and the current study 
(Group IV). Inspection of the data reveals that the improved static 
method did as well as the best previous method on most of the performance 
measures. The results of the second series of ANOVA's are depicted in 
Table 5. Similar to the comparable analysis performed in the previous 
study, this analysis failed to detect significant differences for most 
of the individual measures due to the small sample size. 
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TABLE 5: ‘Analysis of Variance Results (F Ratio) for 

Individual Flight Control Performance Measures 


PERFORMANCE MEASURE 

SOURCE 

TESTS 

METHODS 

TxM 


1 

ALTITUDE ERROR iFTl 

6.047* •• 


0.949 


2 

LATERAL ERROR (FT) 

2.747* 


1.146 

< 

3 

HEADING ERROR (DEG) 

2.448* 

1.401 

0.783 

< 

4 

VELOCITY (KTSl 

1.830 

2.674 * 

2,927** 

H 

6 

DESCENT RATE ERROR 'FT/SECl 

5.479* •• 

3.061* 

2.054* 


6 

INTEGRATED VELOCITY ERROR (FT-SEC) 

1 0.885* •• 

1.629 

3,099** 


7 

ALTITUDE ERROR (FT) 

0.236 

2.342 

1,879 


8 

LATERAL ERROR (FT) 

1.008 

1.025 

1.833 

iu 

9 

HEADING ERROR (DEG) 

0.278 

1.206 

0.477 

a: 

10 

VELOCITY (KTS) 

0.300 

0,142 

0.713 

5 

11 

DESCENT RATE ERROR (FT/SEC) 

2.701 * 

0.782 

3,479* •• 


12 

INTEGRATED VELOCITY ERROR (KT-SEC) 

12.139*** 

1.504 

6.760* •• 


13 

INTEGRATED ALTITUDE ERROR IKT-SEC) 

5,801 ••* 

2.427 

0.923 


14 

INTEGRATED LATERAL ERROR (FT-SEC) 

6.748*** 

2.475* 

0.617 


15 

' ALTITUDE ERROR (FT) 

0.987 

1.187 

2.166* 

O 

.16 

LATERAL ERROR (FT) 

0.915 

3.622** 

2.082* 


17 

HEADING ERROR (DEG) 

1.036 

0.812 

1.036 

X 

18 

VELOCITY (KTS) 

3.578** 

0.354 

0.710 

111 

19 

DESCENT RATE ERROR (FT-SEC) 

0.621 

3.016* 

2.666** 

QC 

X 

20 

INTEGRATED VELOCITY ERROR (KT-SEC) 

3.740** 

0.247 

1.958 

H 

21 

INTEGRATED ALTITUDE ERROR (FT-SEC) 

0.248 

2.213 

2.182* 


22 

INTEGRATED LATERAL ERROR (FT-SEC) 

1.686 

1.708 

- 2.204* 


23 

LATERAL ERROR (FT) 

0.669 

0.776 

2.469** 


24 

DOWN RANGE ERROR (FT) 

2.333 

2.189 

0,738 

z 

25 

heading ERROR (DEG) 

3.388* * 

0.813 

0.679 

5 

26 

VELOCITY (KTS) 

4.303** 

0.334 

?.395 

□ 

27 

DESCENT RATE (FT-SEC) 

6.357*** 

2.981 

2.788** 

X 

o 

28 

bank ANGLE (DEG) ' 

1.235 

0.616 

1.259 

D 

29 

PITCH ANGLE (DEG) 

1 0.026* •• 

0.765 

1.862 


30 

INTEGRATED VELOCITY ERROR (KT-SEC) 

0.987 

1.316 

0.889 


31 

INTEGRATED ALTITUDE ERROR (FT-SEC) 

1.719 

2.142 

2.849** 


32 

INTEGRATED LATERAL ERROR (FT-SEC) 

0.443 

0.823 

0.627 


•P < .10 


^ '"P < .05 

' ' ***P < .10 
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4. DISCUSSION AND CONCLUSIONS 

The effectiveness of dn improved training method was compared iii this 
study with the eft^ecti veness of methods previously investigated (Sitter- 
ley, et al , 1972). Whenever new results are evaluated in relation to 
results previously obtained, the comparability of simulator characteris- 
tics, experimental procedures, and test subjects Is important to the 
validity of the results and conclusions. Since both studies required 
that the test subjects experience the same experimental conditions after 
relatively long intervals of time, considerable care was exercised to 
make certain that the experimental conditions could be closely duplicated 
from the very beginning. 

The aerodynamics model and scaling of the electronic flight instruments 
were held constant by digital computer programs and hardwired circuit 
cards. High fidelity calibration recordings of all flight control ele- 
ments of the simulator were used to maintain the empirical equivalence 
of the display/control operations and the camera servo systems. Detail- 
ed experimental procedure checklists for simulator checkout and operation, 
subject training, data collection, and analysis provided the basis for 
maintaining close control and repeatability of all experimental procedures, 
both within and between studies. 

Likewise, the test subjects were obtained from the same pilot population 
using the same selection criteria for both studies. The time required 
to train to criterion, and the performance at the end of training, amply 
demonstrated that the pilots in the current study were very comparable to 
those used in the previous study. It may be assumed, therefore, that 
the characteristics and fidelity of the simulator and experimental pro- 
cedures as well as subject selection in the current study provided a 
close replication of the previous study. As such, results obtained using 
the advanced static retraining method may be compared with the retrain- 
ing method results of the previous study with reasonable confidence. 
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The results of this study showed that use of the improved static retrain- 
ing method countered skill degradation beyond expectations. After 4 months 
without practice, the improved static method prevented any significant 
or practical degradation in flight control performance during the simula- 
ted approach and landing. Furthermore, and perhaps most significant, the 
improved static retraining method appeared to be more effective in count- 
ering skill degradation than even the dynamic display method, which was 
most effective in the previous study. 

The previous study demonstrated the substantive requirement for critical 
visual cue and flight operation reinforcement for skill retention train- 
ing, and suggested that alternate methods of retraining which do not in- 
volve closed-loop interaction with the pilot may be feasible. The pr >- 
ent study reconfirms these conclusions. It was also suggested that the 
important element in effective retraining was the inclusion of a more 
complete and dynamic representation of the visual fligfit environment. 
However, the success of the improved static method in the present study 
failed to confirm the necessity of dynamic visual cue presentation for 
retraining. Apparently, the carefully structured visual cues of the im- 
proved static method were sufficient to "key" the appropriate pilot re- 
sponses even though they were presented in a static or still form. 

The improved static retraining completely overcame the inadequacies of 
the original static retraining method by providing a more complete stop 
action sequence of events which carefully integrated th'' visual cues, 
flight profile, cockpit instrumental on, and required control responses. 

In addition, the improved method provided the comparative basis for 
recognizing the limits of good performance and the proper recovery from 
off-nominal situations. It is clear that the systematic identification 
of retraining requirements and structuring of the course content formed 
the basis for the current success of static retraining. 

However, it is not so clear why the improved static method was more ef- 
fective than the dynamic display retraining method. With the dynamic 
display method, the pilot sat in the cockpit and viewed the instruments 
and out-the-window visual scene throughout the complete flight. While 
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the pilot did not have the capability to interact with the flight situa- 
tion via the controls, all of the dynamics of instrument operation and 
visual field were available to him. Certainly, the visual cues, in their 
proper content, were more complete in the dynamic display method than 
in the current improved static method retraining. The effectiveness of 
the improved static method does not, therefore, appear to be related to 
the total inclusion of all visual cues or to the presentation of the 
necessary cues in dynamic or motion form. 

;i 

Apparently the important element was the presentation of those cues or 
stimuli which assisted the pilots to recall their basic flight experience, 
both perceptual and. control, and to apply it to the characteristics of ' 
the current flight problem. The flexibility of the self-paced retraining 
approach j coupled. with the integration of the critical instrument, flight 
profile, and visual cues were the unique characteristics of the improved 
static retraining. The graphic self-test further reinforced these charac- 
teristics and the learning situation by providing direction to, and inter- 
active involvement with, the retraining materials. 

Cone! usions 

This study confirms the thesis that properly structured open-loop methods 
of flight control task retraining are feasible. Furthermore, it indicates 
that these retraining methods do not require a dynamic presentation media 
to be effective. 

Application of the results of this study can have a significant impact on 
the cost-effectiveness of recurrent and transition flight training. Fuel 
and aircraft costs are at a premium for flight training. Certainly simu- 
lators can and do relieve a significant portion of this burden. However, 
simulators are still relatively expensive and their widespread availability 
for training is limited. 

Current advancements in the state-of-the-art of training technology may 
well permit an off-loading of simulator training time similar to that 
which simulators have provided for flight training. At a minimum, the 
benefits of improved: self-paced training materials should materially 
enhance the effective utilization of our limited simulator and aircraft 
training hours. 
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